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Introduction 


In the September 201 1 issue of School Library Journal (SLJ), we presented 
a national analysis of National Center for Education Statistics (NCES) state- 
level data on librarian staffing and fourth-grade reading scores from the 
National Assessment of Educational Progress (NAEP). 1 The results from 
this study indicated that states that gained school librarians from 2004-05 to 
2008-09 showed greater increases in 4 th grade reading scores than did 
states that lost librarians during this time period. As promised, we now move 
back to the Colorado context to examine the relationships between changes 
in school library staffing and changes in Colorado Student Assessment 
Program (CSAP) reading scores over time. 

Librarians and Reading 


Before launching into the data, analysis, and findings, let’s consider what 
school librarians believe about the primacy of reading and what they see as 
their role in teaching it. 


Reading is a window to the world. Reading is a foundational skill for 
learning, personal growth, and enjoyment. The degree to which students can 
read and understand text in all formats (e.g., picture, video, print) and all 
contexts is a key indicator of success in school and in life. As a lifelong 
learning skill, reading goes beyond decoding and comprehension to 
interpretation and development of new understandings. 

American Association of School Librarians (AASL), 

Standards for the 21 st Century Learner 



AASL’s Standards for the 21 st Century Learner do not just pay lip-service to 
the value of reading. They have been aligned with the Common Core 
standards that were derived from, and now influence, state standards-based 
tests, such as the CSAP reading tests. 2 It is clear that school librarians 
consider the teaching and encouragement of reading— both for curricular 
reasons and as a lifelong learning skill— to be among their most basic 
responsibilities. Accordingly, they intentionally pursue a wide variety of 
activities that should be expected to have a direct impact on students’ 
reading scores. 

Data Issues 

As noted in the nationally focused SLJ article, our first analysis was 
exploratory and necessarily imprecise, due to the nature of the data 
available. The NCES counts of librarians included all librarians— endorsed 
and non-endorsed— and NAEP scores were limited to a specific grade. At 
the state level, more detailed library staffing and testing data were available 
that allowed us to improve upon this exploratory analysis. Also, we had 


1 Lance, K.C., & Hofschire, L. (201 1 , September 1). Something to shout about: New research shows that 
more librarians means higher reading scores. School Library Journal, 57, 28-33. 

2 See Crosswalk of the Common Core Standards and Standards for the 21 st Century Learner at: 
http://www.ala.org/ala/mgrps/divs/aasl/guidelinesandstandards/commoncorecrosswalk/index.cfm. 


1 Library Research Service 




access to considerably newer data for this study (2011) than for the national 
analysis (2009). Compared with the national analysis, this study required 
handling both staffing and test score data differently to accommodate the 
realities of the data at the local level 


2005 and 2011 

Why did we select 2005 and 201 1 as the focal years of this study? The latter 
year was chosen because it is what is most currently available. As for 2005, 
that was the school year approximately halfway between the recent Great 
Recession and the previous recession. As tightened budgets resulting from 
recession and post-recession fiscal pressures have probably been the major 
factor influencing school library staffing decisions, it seemed important to 
choose a base year as distant as possible from the two latest recessions, 
both of which impacted public schools severely. 

Colorado Data on School Library Staffing and CSAP Reading Scores 

From the Colorado Department of Education (CDE), we obtained school-by- 
school staffing data that included counts of librarians and library assistants, 
both those endorsed and those not endorsed by CDE. 

In Colorado, reading is one of the most prevalent areas of CSAP testing, 
including almost all students in grades 3 to 1 0. Based on their scores, the 
performance of students is categorized as advanced, proficient, partially 
proficient, or unsatisfactory. To maximize the number of cases (i.e., schools) 
in this study, we summarized reading results by school, adding the number 
of each school’s students scoring at the advanced and unsatisfactory levels 
across grades and dividing by the number of students tested in the school. 
Similar testing statistics are made available for schools and districts on 
CDE’s SchoolView® microsite. 3 

Analyzing school-level data across Colorado presented some challenges not 
experienced with the national analysis. Nationally, we looked at percent 
change in the number of school librarians in each state. Because librarians 
were reported in substantial numbers in all states, this worked. Examining 
local data statewide, we found that this would not work, due to the often all- 
or-nothing nature of school library staffing (i.e., some schools had no 
librarians in at least one of the two years in the study). Likewise, when 
examining reading scores nationwide, we looked at percent change in actual 
NAEP reading scores for fourth-graders. Examining local data statewide, we 
found that this would not work, due to the extremes in 201 1 reading 
performance levels and the progress they represented since 2005. What 
was especially problematic was the confounding nature of these two 
elements. For instance, some schools with similar percentages of 
improvement over time had started out at opposite extremes— one 
advanced, the other unsatisfactory— in 2005. 


In Colorado, reading is one 
of the most prevalent areas 
of CSAP testing, including 
almost all students in grades 
3 to 10. Based on their 
scores, the performance of 
students is categorized as 
advanced, proficient, 
partially proficient, or 
unsatisfactory. 


3 http://www.schoolview.org/ 
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To take into account these realities, change over time in both librarian 
staffing and reading performance levels was represented by two categorical 
variables. 

Change in Endorsed Librarian Staffing Matrix 

Disregarding school-to-school variations in full-time staffing counts and 
degrees of part-time status, we found that each school fell into one of four 
categories for librarian staffing trends: 

1 . It had an endorsed librarian in both 2005 and 201 1 . 

2. It had an endorsed librarian in 201 1 , but did not have one in 2005. 

3. It did not have an endorsed librarian in 201 1 , but did have one in 
2005. 

4. It did not have an endorsed librarian in either 2005 or 201 1 . 

Level/ Change for Reading Scores Matrices 

Schools were also classified based on two dimensions related to CSAP 
reading scores: Whether they scored above or below the median for each 
performance level in 2011, and whether the change in their performance 
level between 2005 and 201 1 was above or below the median for such 
change. Hence, each school also fell into one of four categories for level and 
change in reading scores: 

1 . It had higher scores in 201 1 , and a higher increase in scores from 
2005 to 2011. 

2. It had higher scores in 201 1 , and a lower increase in scores from 
2005 to 2011. 

3. It had lower scores in 201 1 , and a higher increase in scores from 
2005 to 2011. 

4. It had lower scores in 201 1 , and a lower increase in scores from 
2005 to 2011. 

Advanced Reading Levels by Endorsed Librarian 
Trends, 2005 to 2011 


Schools that either 
maintained or gained an 
endorsed librarian between 
2005 and 2011 tended to have 
more students scoring 
advanced in reading in 2011 
and to have increased their 
performance more than 
schools that either lost their 
librarians or never had one. 


There is a positive and statistically significant relationship between 
advanced reading levels and endorsed librarian staffing trends (see Chart 
1 -) 

Schools that either maintained or gained an endorsed librarian between 
2005 and 201 1 tended to have more students scoring advanced in reading 
in 201 1 and to have increased their performance more since 2005 (45% and 
49%, respectively) than schools that either lost their librarians or never had 
one ( 33% and 29%). Conversely, schools that either lost a librarian during 
that period or never had one (33% and 39%) tended to have fewer students 
scoring advanced in 201 1 and to have seen lesser gains— or indeed, 
losses— since 2005 than schools that maintained or gained a librarian (23% 
and 18%). 
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Notably, schools with the largest percentage of higher advanced reading 
scores in 201 1 and higher increases in advanced reading scores between 
2005 and 201 1 (49%) were those that gained an endorsed librarian during 
that interval. If an endorsed librarian is doing her or his job well, this is what 
one would expect. 

Unsatisfactory Reading Levels by Endorsed 
Librarian Trends, 2005 to 2011 


There is a negative and statistically significant relationship between 
unsatisfactory reading levels and endorsed librarian staffing trends (see 
Chart 2). 

Schools that either maintained or gained an endorsed librarian between 
2005 and 201 1 tended to have fewer students scoring unsatisfactory in 
reading in 201 1 (i.e., lower scores) (28% and 26%, respectively) and to have 
reduced that problem more since 2005 (i.e., lower increase) than schools 
that either lost their librarians or never had one (both at 34%). Conversely, 
schools that either lost a librarian during this period or never had one (32% 
and 34%) tended to have more students scoring unsatisfactory in 201 1 and 
to have seen that problem increase more since 2005 than schools that 
maintained or gained a librarian (21% and 30%). 

Notably, schools with the largest percentage of lower unsatisfactory reading 
scores in 201 1 and lower increases in that figure between 2005 and 201 1 
(34%) were those that gained an endorsed librarian during the interval. As 
with advanced reading scores, if an endorsed librarian is doing her or his job 
well, this is what one would expect. 


Notably, schools with the 
largest percentage of 
higher advanced reading 
scores in 2011 and 
higher increases in 
advanced reading scores 
between 2005 and 2011 
(49%) were those that 
gained an endorsed 
librarian during that 
interval. 
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Chart 2 

Colorado Public Schools by Unsatisfactory Reading Levels by 
Endorsed Librarian Trends, 2005-2011 

Librarian 2005 & 'll Librarian 2011, not' 05 No librarian 2011, librarian '05 No librarian 2005 & 'll 


0% 

10 % - 
20% 
30% - 
40% 
50% 
60% 
70% 
80% 
90% 
100 % 


24% 


19% 


21% 


Lower scores, lower increase Lower scores, higher increase Higher scores, lower increase Higher scores, higher increase 


The results show that in 
2011, schools with at least 
one FTE endorsed librarian 
averaged significantly 
higher advanced CSAP 
reading scores and 
significantly lower 
unsatisfactory scores than 
schools with less than one 
FTE endorsed librarian. 


Assessing Three Alternative Models of School 
Library Staffing by Association with CSAP 
Reading Scores, 2011 

In the above analysis of change in librarian staffing and reading scores from 
2005 to 201 1 , we see the consequences over time of schools shifting from 
one library staffing model to another. Basically, there are three major options 
for who runs a school library: an endorsed librarian (i.e., endorsed by CDE 
as either a School Librarian or a Teacher Librarian/Media Specialist), a non- 
endorsed librarian (i.e., having neither type of librarian endorsement), and a 
non-endorsed library assistant. Many school libraries have library assistants, 
ideally working under the supervision of an endorsed librarian. In reality, 
however, there are more and more schools— especially at the elementary 
level— where library assistants run libraries without supervision. It is the type 
of situation that would never be tolerated in the classroom teaching context 
at any level. Teacher aides rarely if ever work independently of teachers. 

To shed further light on the relative merits of these three library staffing 
options, we examined 2011 data alone, comparing mean percentages of 
students earning advanced and unsatisfactory CSAP reading scores for 
schools with at least one full-time equivalent (FTE) endorsed librarian to 
schools with less than one FTE of this type. Similar analyses were 
conducted for non-endorsed librarians and non-endorsed library assistants. 
The results, illustrated in Chart 3, show that in 201 1 , schools with at least 
one FTE endorsed librarian averaged significantly higher advanced CSAP 
reading scores (8% vs. 6%) and significantly lower unsatisfactory scores 
(9% vs. 11%) than schools with less than one FTE endorsed librarian. 

Differences in 201 1 advanced and unsatisfactory reading scores for schools 
with and without non-endorsed librarians and with and without non-endorsed 
library assistants (working without endorsed or non-endorsed librarians) 
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were not statistically significant. In other words, school library programs that 
were not managed by endorsed librarians— whether the alternative was a 
non-endorsed librarian or library assistant— had no measurable association 
with CSAP reading scores. 
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Chart 3 

Colorado Public Schools Scoring Advanced & Unsatisfactory in 
Reading by Library Staffing Level, 2011 
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Taking Poverty Into Account 


Whenever studies of school libraries report associations between levels of 
library staffing and test scores, a very reasonable question is posed: Why do 
you believe this association reflects cause-and-effect? Perhaps, the 
questioner suggests, it is simply a matter of schools in wealthier 
communities being able to afford to employ endorsed librarians and parents 
in wealthier communities raising children who— being more advantaged 
socially and economically, and living in more information-rich home 
environments— tend to earn higher test scores. Indeed many of the school 
library impact studies conducted since 2000 found that the strongest 
predictor of test scores was poverty or the lack thereof in the community. 
Those studies also found that school libraries were still associated with test 
scores, even when poverty was taken into account. 

To take it into account in this study, we conducted a partial correlation 
analysis of library staffing level and advanced and unsatisfactory reading 
levels, controlling for the percentage of students in a school who were 
eligible for Free and Reduced-Cost Meals (the generally accepted indicator 
of poverty in education research). In other words, this type of analysis 
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allowed us to determine whether librarian staffing still had an impact on 
CSAP scores, even when holding poverty constant. Our results differed 
slightly from the staffing model analyses discussed above. In this instance, 
both endorsed and non-endorsed librarians were positively correlated with 
advanced CSAP reading scores and negatively correlated with 
unsatisfactory scores. In other words, with poverty utilized as a control 
variable, both endorsed and non-endorsed librarians had positive and 
statistically significant correlations with reading scores. Notably, however, 
these relationships are stronger for endorsed librarians than non-endorsed 
ones. What did not change was the lack of relationship between non- 
endorsed library assistants working without a librarian and reading scores. 
Apparently, library assistants working without supervision do not have any 
impact on reading scores, either advanced or unsatisfactory. 

Table 1. Partial Correlation Coefficients for CSAP Reading 
Achievement and Library Staffing Levels Controlling for Poverty for 
Colorado Public Schools, 2011 


Percent of 
Students by 

CSAP Reading 
Level 

O — 

)l Library Staffing Level in 
'ime Equivalents (FTEs) 

Endorsed 
librarian 
(N= 1548) 

Non-endorsed 
librarian 
(N=1 048) 

Non-endorsed 
library assistant 
w/no librarian 
(N=766) 

Advanced 

.23* 

.17* 

.06 

Unsatisfactory 

-.12* 

-.09** 

-.02 


*p<001;**p<05 


Notes to Table 1: 

1 . N represents the number of cases (schools) for which data were 
available, not the number of schools with each type of school 
library staff. 

2. The standard of statistical significance is .01 , indicating less than 
one chance in 100 that the results occurred randomly. Notably, for 
all but one of the significant findings, there was less than one 
chance in 1,000 of such a random occurrence. 

3. The analysis for non-endorsed library assistants excluded schools 
with endorsed and non-endorsed librarians. Thus, the correlation 
coefficients for this staffing level are not comparable to the others. 

4. Poverty was represented by the percentage of a school’s students 
who were eligible for Free and Reduced-Cost Meals. 

Conclusion 


The findings of this Colorado analysis update, confirm, and extend the 
findings of our recent SLJ article. Similar to the results presented in that 
article, we found that students at schools that gained or maintained an 
endorsed librarian to manage the library program averaged higher CSAP 
reading scores and higher increases in those scores over time than students 
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at schools whose library programs were run by either non-endorsed 
librarians or library assistants. In the national analysis, it was not possible to 
distinguish between endorsed and non-endorsed librarians; so, these 
findings about the value of gaining or maintaining an endorsed librarian 
when school budgets get tight are new. As in earlier state-level school library 
impact studies and the SLJ national study, the association of endorsed 
librarians with higher reading scores cannot be explained away by local 
economic conditions. 

The findings of these two studies could be extended in many ways. Here are 
a few ideas: 

1 . Colorado scores for subjects other than reading could be 
examined. Both writing and science are interesting prospects. 
Unfortunately, there are many subjects for which there are no state 
test scores. An especially challenging question for researchers of 
school library impact is “what data do we have, or could we create, 
to study student learning in other areas?” 

2. Similar analyses could be conducted in other states. Mandated by 
federal and state governments, the data required to replicate this 
study are available in every state, so such studies do not have to 
be prohibitively expensive. So far, we have two studies: the 
national analysis published in SLJ and this state one. Similar 
studies should be done in a variety of different states around the 
U.S., as happened with the original Colorado-style impact studies. 

If similar findings can be reported from disparate settings, findings 
of this type will have more credence. 

3. Qualitative studies could examine the very intriguing question of 
how teachers and students cope with the absence of an endorsed 
librarian or, in some cases, even an entire library program. As the 
data used in this study indicate, some schools manage to have 
successful students, despite what appear to be inadequately 
staffed library programs. They are not the norm, and certainly not 
the ideal; but, such schools do exist. Their existence raises many 
questions: 

• How are these schools managing to meet their students’ and 
teachers’ needs, for both information resources and guidance in 
accessing and using them, short- and long-term? 

• What alternatives are being employed in these schools? Do they 
rely heavily on online resources, alone or in combination with 
something else? 
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• Who guides the use of such resources: technology teachers, 
classroom teachers, or perhaps one special individual on the 
school’s faculty or staff who is a de facto librarian? 

• Or, do such schools have special relationships with the public 
and/or academic libraries in their communities that compensate for 
the lack of a strong school library program? 

These are only a few of many possible questions about this phenomenon. 

To our knowledge, there have been no studies of these unusual cases. If we 
are to understand how school library programs are evolving, and may need 
to evolve, such studies are needed. It is not a matter of discovering who is 
“taking over’’ from school librarians. Perhaps we will discover a new niche in 
the school labor force that school librarians can occupy. 

For the time being, what we know from this study and its predecessors is 
that the research on school librarians and their association with students’ 
test scores is remarkably consistent in its findings: regardless of how rich or 
poor a community is, students tend to perform better on reading tests where, 
and when, their library programs are in the hands of endorsed librarians. 
Furthermore, at schools where library programs gain or maintain an 
endorsed librarian when school budgets get tight, students tend to excel. At 
schools where library programs lose or never had an endorsed librarian, 
students suffer as a result. 
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Appendix A 


The following tables provide detailed results of the statistical analyses reported in this article. 


Table A-1. Colorado Public Schools by Advanced Reading Levels by Endorsed Librarian Trends, 2005-2011 



Endorsed Librarian Change, 2005-1 1 





No librarian 



Advanced CSAP 


Librarian 

2011, 



Reading Scores 2011 

Librarian 

2011, not 

librarian 

No librarian 


& 2005-2011 

2005 & 2011 

2005 

2005 

2005 & 2011 

Total 

Higher score, higher 

128 

81 

34 

210 

453 

increase 

44.9% 

49.1% 

33.3% 

29.2% 

35.6% 

Higher score, lower 

57 

30 

15 

128 

230 

increase 

20.0% 

18.2% 

14.7% 

17.8% 

18.1% 

Lower score, higher 

35 

24 

19 

105 

183 

increase 

12.3% 

14.5% 

18.6% 

14.6% 

14.4% 

Lower score, lower 

65 

30 

34 

277 

406 

increase 

22.8% 

18.2% 

33.3% 

38.5% 

31.9% 


285 

165 

102 

720 

1272 

1 0131 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 


Chi-square = 54.243, p = .000 


Table A-2. Colorado Public Schools by Unsatisfactory Reading Levels by Endorsed Librarian Trends, 2005-2011 


Unsatisfactory CSAP 
Reading Scores 2011 & 
2005-2011 

Endorsed Librarian Change, 2005-11 

Total 

Librarian 2005 & 
2011 

Librarian 2011, 
not 2005 

No librarian 
2011, librarian 
2005 

No librarian 
2005 & 2011 

Higher score, higher 

78 

43 

39 

253 

413 

increase 

27.6% 

26.1% 

33.9% 

33.5% 

31.3% 

Higher score, lower 

46 

31 

30 

155 

262 

increase 

16.3% 

18.8% 

26.1% 

20.5% 

19.9% 

Lower score, higher 

69 

35 

22 

123 

249 

increase 

24.4% 

21.2% 

19.1% 

16.3% 

18.9% 

Lower score, lower 

90 

56 

24 

225 

395 

increase 

31.8% 

33.9% 

20.9% 

29.8% 

29.9% 


283 

165 

115 

756 

1319 

1 0131 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

Chi-square = 20.564, p = .01 
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Table A-3 (Part 1). Comparison-of-Means Analysis (t Test for Independent Samples) for Percent of Students with 


CSAP Reading Level 

Endorsed Librarian FTE 

N 

Mean 

Standard Deviation 

Standard Error 
Mean 

Advanced 





>= 1.0000 

405 

.0814 

.06524 

.00324 

< 1.0000 

1144 

.0565 

.05421 

.00160 

Unsatisfactory 





>= 1.0000 

405 

.0899 

.07765 

.00386 

< 1.0000 

1144 

.1060 

.08996 

.00266 


Advanced and Unsatisfactory Reading Scores by Endorsed Librarian Staffing, 2011 


Table A-3 (Part 2) 






Mean 

Standard error 

95% Confidence Interval 


t 

df 

P 

difference 

difference 

Lower 

Upper 

Advanced 

6.874 

612.695 

.000 

.02486 

.00362 

.01776 

.03196 

Unsatisfactory 

-3.426 

814.093 

.001 

-.01606 

.00469 

-.02525 

-.00686 


Advanced F = 16.786, p = .000 
Unsatisfactory F = 1 1 .412, p = .001 


Table A-4 (Part 1). Comparison-of-Means Analysis (t Test for Independent Samples) for Percent of Students with 
Advanced and Unsatisfactory Reading Scores by Non-Endorsed Librarian Staffing, 2011 


CSAP Reading Level 

Non-Endorsed Librarian FTE 

N 

Mean 

Standard Deviation 

Standard Error 
Mean 

Advanced 





>= 1.0000 

195 

.0698 

.05595 

.00401 

< 1.0000 

1354 

.0620 

.05860 

.00159 

Unsatisfactory 

>= 1.0000 

195 

.0957 

.07163 

.00513 

< 1.0000 

1354 

.1027 

.08917 

.00242 


Table A-4 (Part 2). 



t 

df 

P 

Mean 

difference 

Standard error 
difference 

95% Confidence Interval 

Lower 

Upper 

Advanced 

1.749 

1547 

.080 

.00781 

.00446 

-.00095 

.01656 

Unsatisfactory 

-1.220 

288.207 

.224 

-.00692 

.00567 

-.02002 

.00425 


Advanced F = .012, p = .913 
Unsatisfactory F = 7.322, p = .007 
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Table A-5 (Part 1). Comparison-of-Means Analysis (t Test for Independent Samples) for Percent of Students with 
Advanced and Unsatisfactory Reading Scores by Non-Endorsed Library Assistant Staffing, 2011 


CSAP Reading Level 
Non-Endorsed Library Assistant 

FTE 

N 

Mean 

Standard Deviation 

Standard Error 
Mean 

Advanced 





>= 1.0000 

262 

.0455 

.04416 

.00273 

< 1.0000 

507 

.0505 

.05080 

.00226 

Unsatisfactory 





>= 1.0000 

262 

.1198 

.09311 

.00575 

< 1.0000 

507 

.1107 

.09849 

.00437 


Table A-5 (Part 2). 






Mean 

Standard error 

95% Confidence Interval 


t 

df 

P 

difference 

difference 

Lower 

Upper 

Advanced 

-1.416 

596.179 

.157 

-.00501 

.00354 

-.01196 

.00194 

Unsatisfactory 

1.244 

767 

.214 

.00915 

.00736 

-.00529 

.02359 


Advanced F = 1 1 .078, p = .001 
Unsatisfactory F = .477, p = .490 
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